{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Named entity recognition with OpenVINO™\n",
    "\n",
    "\n",
    "The Named Entity Recognition(NER) is a natural language processing method that involves the detecting of key information in the unstructured text and categorizing it into pre-defined categories. These categories or named entities refer to the key subjects of text, such as names, locations, companies and etc.\n",
    "\n",
    "NER is a good method for the situations when a high-level overview of a large amount of text is needed. NER can be helpful with such task as analyzing key information in unstructured text or automates the information extraction of large amounts of data.\n",
    "\n",
    "\n",
    "This tutorial shows how to perform named entity recognition using OpenVINO. We will use the pre-trained model [`elastic/distilbert-base-cased-finetuned-conll03-english`](https://huggingface.co/elastic/distilbert-base-cased-finetuned-conll03-english). It is DistilBERT based model, trained on `conll03 english dataset`. The model can recognize four named entities in text: persons, locations, organizations and names of miscellaneous entities that do not belong to the previous three groups. The model is sensitive to capital letters.\n",
    "\n",
    "To simplify the user experience, the [Hugging Face Optimum](https://huggingface.co/docs/optimum) library is used to convert the model to OpenVINO™ IR format and quantize it.\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "### Installation Instructions\n",
    "\n",
    "This is a self-contained example that relies solely on its own code.\n",
    "\n",
    "We recommend  running the notebook in a virtual environment. You only need a Jupyter server to start.\n",
    "For details, please refer to [Installation Guide](https://github.com/openvinotoolkit/openvino_notebooks/blob/latest/README.md#-installation-guide).\n",
    "\n",
    "<img referrerpolicy=\"no-referrer-when-downgrade\" src=\"https://static.scarf.sh/a.png?x-pxid=5b5a4db0-7875-4bfb-bdbd-01698b5b1a77&file=notebooks/named-entity-recognition/named-entity-recognition.ipynb\" />\n",
    "\n",
    "#### Table of contents:\n",
    "\n",
    "- [Prerequisites](#Prerequisites)\n",
    "- [Download the NER model](#Download-the-NER-model)\n",
    "- [Select inference device](#Select-inference-device)\n",
    "- [Running OpenVINO model inference](#Running-OpenVINO-model-inference)\n",
    "- [Running OpenVINO model inference via Optimum Intel](#Running-OpenVINO-model-inference-via-Optimum-Intel)\n",
    "- [Quantize the model, using Hugging Face Optimum API](#Quantize-the-model,-using-Hugging-Face-Optimum-API)\n",
    "- [Compare the Original and Quantized Models](#Compare-the-Original-and-Quantized-Models)\n",
    "    - [Compare performance](#Compare-performance)\n",
    "    - [Compare size of the models](#Compare-size-of-the-models)\n",
    "- [Prepare demo for Named Entity Recognition OpenVINO Runtime](#Prepare-demo-for-Named-Entity-Recognition-OpenVINO-Runtime)\n",
    "\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prerequisites\n",
    "[back to top ⬆️](#Table-of-contents:)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import platform\n",
    "\n",
    "%pip uninstall -q -y optimum optimum-intel optimum-onnx\n",
    "%pip install -q \"diffusers>=0.17.1,<0.35.0\" \"openvino>=2025.1.0\" \"openvino-tokenizers\" \"nncf>=2.16.0\" \"gradio>=4.19\" \"transformers==4.53.3\" \"torch==2.8\" \"torchvision==0.23.0\" \"datasets<4.0.0\" --extra-index-url https://download.pytorch.org/whl/cpu\n",
    "%pip install -q \"git+https://github.com/huggingface/optimum-intel.git\"\n",
    "\n",
    "if platform.system() == \"Darwin\":\n",
    "    %pip install -q \"numpy<2.0\""
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Download the NER model\n",
    "[back to top ⬆️](#Table-of-contents:)\n",
    "\n",
    "We load the [`distilbert-base-cased-finetuned-conll03-english`](https://huggingface.co/elastic/distilbert-base-cased-finetuned-conll03-english) model from the [Hugging Face Hub](https://huggingface.co/models) with [Hugging Face Transformers library](https://huggingface.co/docs/transformers/index)and Optimum Intel with OpenVINO integration. For model conversion we will use `optimum-cli` tool."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pathlib import Path\n",
    "\n",
    "import requests\n",
    "\n",
    "if not Path(\"notebook_utils.py\").exists():\n",
    "    r = requests.get(\n",
    "        url=\"https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/latest/utils/notebook_utils.py\",\n",
    "    )\n",
    "    open(\"notebook_utils.py\", \"w\").write(r.text)\n",
    "\n",
    "if not Path(\"cmd_helper.py\").exists():\n",
    "    r = requests.get(\n",
    "        url=\"https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/latest/utils/cmd_helper.py\",\n",
    "    )\n",
    "    open(\"cmd_helper.py\", \"w\").write(r.text)\n",
    "\n",
    "# Read more about telemetry collection at https://github.com/openvinotoolkit/openvino_notebooks?tab=readme-ov-file#-telemetry\n",
    "from notebook_utils import collect_telemetry\n",
    "\n",
    "collect_telemetry(\"named-entity-recognition.ipynb\")\n",
    "\n",
    "from cmd_helper import optimum_cli\n",
    "\n",
    "original_ner_model_dir = Path(\"original_ner_model\")\n",
    "\n",
    "model_id = \"elastic/distilbert-base-cased-finetuned-conll03-english\"\n",
    "\n",
    "if not original_ner_model_dir.exists():\n",
    "    optimum_cli(model_id, original_ner_model_dir)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Select inference device\n",
    "[back to top ⬆️](#Table-of-contents:)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f8f9441945794f949e84109564504f60",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Dropdown(description='Device:', index=1, options=('CPU', 'AUTO'), value='AUTO')"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from notebook_utils import device_widget\n",
    "\n",
    "device = device_widget(exclude=\"NPU\")\n",
    "\n",
    "device"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Running OpenVINO model inference\n",
    "[back to top ⬆️](#Table-of-contents:)\n",
    "\n",
    "\n",
    "We can use converted model using native OpenVINO API without any dependency on PyTorch and Transformers code. For data preprocessing we will use OpenVINO Tokenizers.\n",
    "\n",
    "[OpenVINO Tokenizers](https://github.com/openvinotoolkit/openvino_tokenizers) is an OpenVINO extension and a Python library designed to streamline tokenizer conversion for seamless integration into your projects. It supports Python and C++ environments and is compatible with all major platforms: Linux, Windows, and MacOS. You can find more examples OpenVINO Tokenizers usage in this [notebook](../openvino-tokenizers/openvino-tokenizers.ipynb) \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/ea/work/py311/lib/python3.11/site-packages/openvino/runtime/__init__.py:10: DeprecationWarning: The `openvino.runtime` module is deprecated and will be removed in the 2026.0 release. Please replace `openvino.runtime` with `openvino`.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Alex - B-PER\n",
      "Paris - B-LOC\n"
     ]
    }
   ],
   "source": [
    "import openvino as ov\n",
    "import openvino_tokenizers  # noqa: F401\n",
    "import numpy as np\n",
    "\n",
    "core = ov.Core()\n",
    "ov_model = core.compile_model(original_ner_model_dir / \"openvino_model.xml\", device.value)\n",
    "ov_tokenizer = core.compile_model(original_ner_model_dir / \"openvino_tokenizer.xml\")\n",
    "ov_detokenizer = core.compile_model(original_ner_model_dir / \"openvino_detokenizer.xml\")\n",
    "\n",
    "inputs = ov_tokenizer([\"Hello, my name is Alex and I live in Paris\"])\n",
    "logits = ov_model({\"input_ids\": inputs[\"input_ids\"], \"attention_mask\": inputs[\"attention_mask\"]})[0]\n",
    "\n",
    "predictions = np.argmax(logits, axis=2)\n",
    "input_tokens = ov_detokenizer(inputs[\"input_ids\"].reshape(-1, 1))[0]\n",
    "\n",
    "id2label = {0: \"O\", 1: \"B-PER\", 2: \"I-PER\", 3: \"B-ORG\", 4: \"I-ORG\", 5: \"B-LOC\", 6: \"I-LOC\", 7: \"B-MISC\", 8: \"I-MISC\"}\n",
    "\n",
    "for token, label in zip(input_tokens, predictions[0]):\n",
    "    if label != 0:\n",
    "        label_name = id2label[label]\n",
    "        print(f\"{token} - {label_name}\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Running OpenVINO model inference via Optimum Intel\n",
    "[back to top ⬆️](#Table-of-contents:)\n",
    "\n",
    "\n",
    "`OVModelForTokenClassification` is represent model class for Named Entity Recognition task in Optimum Intel. Model class initialization starts with calling `from_pretrained` method. Usage `OVModelForTokenClassification` allows to run model using similar API like original Transformers model.\n",
    "\n",
    "Additionally, Optimum Intel model is compatible with HuggingFace [pipelines](https://huggingface.co/docs/transformers/en/main_classes/pipelines) API."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2025-04-15 12:51:22.761184: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.\n",
      "2025-04-15 12:51:22.775868: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:477] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered\n",
      "WARNING: All log messages before absl::InitializeLog() is called are written to STDERR\n",
      "E0000 00:00:1744707082.790169  958965 cuda_dnn.cc:8310] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered\n",
      "E0000 00:00:1744707082.794673  958965 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered\n",
      "2025-04-15 12:51:22.812038: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.\n",
      "To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Alex - B-PER\n",
      "Paris - B-LOC\n"
     ]
    }
   ],
   "source": [
    "from transformers import AutoTokenizer\n",
    "from optimum.intel import OVModelForTokenClassification\n",
    "import torch\n",
    "\n",
    "tokenizer = AutoTokenizer.from_pretrained(model_id)\n",
    "model = OVModelForTokenClassification.from_pretrained(original_ner_model_dir, device=device.value)\n",
    "\n",
    "inputs = tokenizer(\"Hello, my name is Alex and I live in Paris\", return_tensors=\"pt\")\n",
    "\n",
    "logits = model(**inputs).logits\n",
    "tokens = tokenizer.batch_decode(inputs.input_ids.reshape(-1, 1))\n",
    "\n",
    "predictions = torch.argmax(logits, dim=2)\n",
    "\n",
    "for token, label in zip(input_tokens, predictions[0]):\n",
    "    if label != 0:\n",
    "        label_name = id2label[label.item()]\n",
    "        print(f\"{token} - {label_name}\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Quantize the model, using Hugging Face Optimum API\n",
    "[back to top ⬆️](#Table-of-contents:)\n",
    "\n",
    "Post-training static quantization introduces an additional calibration step where data is fed through the network in order to compute the activations quantization parameters. For quantization it will be used [Hugging Face Optimum Intel API](https://huggingface.co/docs/optimum/intel/index).\n",
    "\n",
    "To handle the NNCF quantization process we use class [`OVQuantizer`](https://huggingface.co/docs/optimum/intel/reference_ov#optimum.intel.OVQuantizer). The quantization with Hugging Face Optimum Intel API contains the next steps:\n",
    "* Model class initialization starts with calling `from_pretrained()` method.\n",
    "* Next we create calibration dataset with `get_calibration_dataset()` to use for the post-training static quantization calibration step. \n",
    "* After we quantize a model and save the resulting model in the OpenVINO IR format to save_directory with `quantize()` method. \n",
    "* Then we load the quantized model. The Optimum Inference models are API compatible with Hugging Face Transformers models and we can just replace `AutoModelForXxx` class with the corresponding `OVModelForXxx` class. So we use `OVModelForTokenClassification` to load the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "test_replace": {
     "num_samples=100,": "num_samples=10,"
    }
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "b14b2069a3c94631adae48670c0b2869",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "README.md:   0%|          | 0.00/12.3k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "c6a418bdc6bb4b558c26b44564ee589b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "conll2003.py:   0%|          | 0.00/9.57k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "99115728ba7d4e0c8f45aa1308c12430",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading data:   0%|          | 0.00/983k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "aa676eefd46d464d9d605262f52a1864",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Generating train split:   0%|          | 0/14041 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "135b01ed4aa34ac78823ca41761f0cf8",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Generating validation split:   0%|          | 0/3250 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "7538b0a34110454fbfdf82b58a19ab58",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Generating test split:   0%|          | 0/3453 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "44ce8385e7934b36889614beba1ec74e",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/100 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "28a259a4c3c34e3aaf0a59c661bbabd8",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Output()"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"></pre>\n"
      ],
      "text/plain": []
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d058b3c141ab48bbaf483a99d918f967",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Output()"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"></pre>\n"
      ],
      "text/plain": []
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "6768402963504600b251ad3037bd91df",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Output()"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"></pre>\n"
      ],
      "text/plain": []
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f802d1cf73ce46efa90dba7e52ddced7",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Output()"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"></pre>\n"
      ],
      "text/plain": []
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from functools import partial\n",
    "from optimum.intel import OVQuantizer, OVConfig, OVQuantizationConfig\n",
    "\n",
    "\n",
    "def preprocess_fn(data, tokenizer):\n",
    "    examples = []\n",
    "    for data_chunk in data[\"tokens\"]:\n",
    "        examples.append(\" \".join(data_chunk))\n",
    "\n",
    "    return tokenizer(examples, padding=True, truncation=True, max_length=128)\n",
    "\n",
    "\n",
    "# The directory where the quantized model will be saved\n",
    "quantized_ner_model_dir = Path(\"quantized_ner_model\")\n",
    "\n",
    "if not quantized_ner_model_dir.exists():\n",
    "    quantizer = OVQuantizer.from_pretrained(model)\n",
    "    calibration_dataset = quantizer.get_calibration_dataset(\n",
    "        \"conll2003\",\n",
    "        preprocess_function=partial(preprocess_fn, tokenizer=tokenizer),\n",
    "        num_samples=100,\n",
    "        dataset_split=\"train\",\n",
    "        preprocess_batch=True,\n",
    "        trust_remote_code=True,\n",
    "    )\n",
    "\n",
    "    # Apply static quantization and save the resulting model in the OpenVINO IR format\n",
    "    ov_config = OVConfig(quantization_config=OVQuantizationConfig(num_samples=len(calibration_dataset)))\n",
    "    quantizer.quantize(\n",
    "        calibration_dataset=calibration_dataset,\n",
    "        save_directory=quantized_ner_model_dir,\n",
    "        ov_config=ov_config,\n",
    "    )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f1020d70684d4ca6bc9658d85834185d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Dropdown(description='Device:', index=1, options=('CPU', 'AUTO'), value='AUTO')"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from notebook_utils import device_widget\n",
    "\n",
    "device = device_widget()\n",
    "\n",
    "device"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load the quantized model\n",
    "optimized_model = OVModelForTokenClassification.from_pretrained(quantized_ner_model_dir, device=device.value)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Compare the Original and Quantized Models\n",
    "[back to top ⬆️](#Table-of-contents:)\n",
    "\n",
    "Compare the original [`distilbert-base-cased-finetuned-conll03-english`](https://huggingface.co/elastic/distilbert-base-cased-finetuned-conll03-english) model with quantized and converted to OpenVINO IR format models to see the difference."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Compare performance\n",
    "[back to top ⬆️](#Table-of-contents:)\n",
    "\n",
    "As the Optimum Inference models are API compatible with Hugging Face Transformers models, we can just use `pipleine()` from [Hugging Face Transformers API](https://huggingface.co/docs/transformers/index) for inference."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Device set to use cpu\n",
      "Device set to use cpu\n"
     ]
    }
   ],
   "source": [
    "from transformers import pipeline\n",
    "import torch\n",
    "import platform\n",
    "\n",
    "additional_args = {}\n",
    "\n",
    "if platform.processor() == \"arm\":\n",
    "    additional_args = {\"device\": torch.device(\"cpu\")}\n",
    "\n",
    "ner_pipeline_optimized = pipeline(\"token-classification\", model=optimized_model, tokenizer=tokenizer, **additional_args)\n",
    "\n",
    "ner_pipeline_original = pipeline(\"token-classification\", model=model, tokenizer=tokenizer, **additional_args)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Median inference time of quantized model: 0.00587007449939847 \n",
      "Median inference time of original model: 0.0049262153916060925 \n"
     ]
    }
   ],
   "source": [
    "import time\n",
    "import numpy as np\n",
    "\n",
    "\n",
    "def calc_perf(ner_pipeline):\n",
    "    inference_times = []\n",
    "\n",
    "    for data in calibration_dataset:\n",
    "        text = \" \".join(data[\"tokens\"])\n",
    "        start = time.perf_counter()\n",
    "        ner_pipeline(text)\n",
    "        end = time.perf_counter()\n",
    "        inference_times.append(end - start)\n",
    "\n",
    "    return np.median(inference_times)\n",
    "\n",
    "\n",
    "print(f\"Median inference time of quantized model: {calc_perf(ner_pipeline_optimized)} \")\n",
    "\n",
    "print(f\"Median inference time of original model: {calc_perf(ner_pipeline_original)} \")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Compare size of the models\n",
    "[back to top ⬆️](#Table-of-contents:)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Size of original model in Bytes is 260795556\n",
      "Size of quantized model in Bytes is 65802752\n"
     ]
    }
   ],
   "source": [
    "from pathlib import Path\n",
    "\n",
    "fp_model_file = Path(original_ner_model_dir) / \"openvino_model.bin\"\n",
    "print(f\"Size of original model in Bytes is {fp_model_file.stat().st_size}\")\n",
    "print(f'Size of quantized model in Bytes is {Path(quantized_ner_model_dir, \"openvino_model.bin\").stat().st_size}')"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prepare demo for Named Entity Recognition OpenVINO Runtime\n",
    "[back to top ⬆️](#Table-of-contents:)\n",
    "\n",
    "Now, you can try NER model on own text. Put your sentence to input text box, click Submit button, the model label the recognized entities in the text."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [],
    "test_replace": {
     "demo.launch(debug=True)": "demo.launch()",
     "demo.launch(share=True, debug=True)": "demo.launch(share=True)"
    }
   },
   "outputs": [],
   "source": [
    "if not Path(\"gradio_helper.py\").exists():\n",
    "    r = requests.get(\n",
    "        url=\"https://raw.githubusercontent.com/openvinotoolkit/openvino_notebooks/latest/notebooks/magika-content-type-recognition/gradio_helper.py\"\n",
    "    )\n",
    "    open(\"gradio_helper.py\", \"w\").write(r.text)\n",
    "\n",
    "from gradio_helper import make_demo\n",
    "\n",
    "demo = make_demo(ner_pipeline_optimized)\n",
    "\n",
    "try:\n",
    "    demo.launch(debug=True)\n",
    "except Exception:\n",
    "    demo.launch(share=True, debug=True)\n",
    "# if you are launching remotely, specify server_name and server_port\n",
    "# demo.launch(server_name='your server name', server_port='server port in int')\n",
    "# Read more in the docs: https://gradio.app/docs/"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  },
  "openvino_notebooks": {
   "imageUrl": "",
   "tags": {
    "categories": [
     "Model Demos"
    ],
    "libraries": [],
    "other": [],
    "tasks": [
     "Named Entity Recognition",
     "Token Classification"
    ]
   }
  },
  "vscode": {
   "interpreter": {
    "hash": "1c707170576399eaaed0c4f2e01a2d1b61ba791ba1842c47e5b3e4f6f79b82ab"
   }
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {
     "08c6f36cda9f40e6a829764815a5d042": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "091e9d9766ce414f98bb52001a874d84": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "0937816eac10417596e1337304c46a34": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "0b0d2679c13648ca823a7728f52ed20c": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "1122e48971774d0db5bb72bdcb362a33": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "135b01ed4aa34ac78823ca41761f0cf8": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HBoxModel",
      "state": {
       "children": [
        "IPY_MODEL_87bf70d51a0d4c229e25ac179e3c3303",
        "IPY_MODEL_517101c6212f46ea881fff19fd840f07",
        "IPY_MODEL_aefdd06f6777415c8eb76e53a479fd66"
       ],
       "layout": "IPY_MODEL_31d25f7e566d4b5089c1b25b0b9ee9c5"
      }
     },
     "162f22edce4446ae8ee2ad3c7e6afb98": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "163cb33d39c444a09f7d99c2ffa93f4d": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "1ffe50db1bab4cd3a76ffce73d96755b": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "212d64bbafdb4b37af03df36843297e5": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLModel",
      "state": {
       "layout": "IPY_MODEL_89af588477f54df7b5bb02f5fd24e8f9",
       "style": "IPY_MODEL_9a876a2e31614fb29efcc8abae021382",
       "value": "Generating test split: 100%"
      }
     },
     "213c849d49074d2784eb4e9a749be116": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "description_width": ""
      }
     },
     "241ee2f446604688a29d27da13e94eba": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "251853a90016490abb9747f3b8d41c28": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "28a259a4c3c34e3aaf0a59c661bbabd8": {
      "model_module": "@jupyter-widgets/output",
      "model_module_version": "1.0.0",
      "model_name": "OutputModel",
      "state": {
       "layout": "IPY_MODEL_534443f19f4f4304964831cda942efa8",
       "outputs": [
        {
         "data": {
          "text/html": "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Statistics collection <span style=\"color: #729c1f; text-decoration-color: #729c1f\">━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━</span> <span style=\"color: #800080; text-decoration-color: #800080\">100%</span> <span style=\"color: #0068b5; text-decoration-color: #0068b5\">100/100</span> • <span style=\"color: #0068b5; text-decoration-color: #0068b5\">0:00:01</span> • <span style=\"color: #0068b5; text-decoration-color: #0068b5\">0:00:00</span>\n</pre>\n",
          "text/plain": "Statistics collection \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[38;2;0;104;181m100/100\u001b[0m • \u001b[38;2;0;104;181m0:00:01\u001b[0m • \u001b[38;2;0;104;181m0:00:00\u001b[0m\n"
         },
         "metadata": {},
         "output_type": "display_data"
        }
       ]
      }
     },
     "2c964cf3312e4ab5800cce91c5f51ed4": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLModel",
      "state": {
       "layout": "IPY_MODEL_0b0d2679c13648ca823a7728f52ed20c",
       "style": "IPY_MODEL_ba3e07f964b140e09a1234d74dac1524",
       "value": "conll2003.py: 100%"
      }
     },
     "2e4b55f1c6d54fa8a6a40d1f8ccdfe52": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "2f38221907b44d5db9af4896d07f21a4": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLStyleModel",
      "state": {
       "description_width": "",
       "font_size": null,
       "text_color": null
      }
     },
     "2f4f1cee64e444c59655032f355f837b": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "FloatProgressModel",
      "state": {
       "bar_style": "success",
       "layout": "IPY_MODEL_bbcb27fb663a42a1980c29ef7b0c7db2",
       "style": "IPY_MODEL_5277e4bda0dd420ab87723f000fff2a8",
       "value": 100
      }
     },
     "2f8a65b6e630498893ac2f3655a70b89": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "description_width": ""
      }
     },
     "3033be28482f4b8c90ead9f18345de66": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "FloatProgressModel",
      "state": {
       "bar_style": "success",
       "layout": "IPY_MODEL_8a93dd201c0440419f43ef30cce70751",
       "max": 982975,
       "style": "IPY_MODEL_9398e11d0c1744f18c35c3a13c7621d7",
       "value": 982975
      }
     },
     "31d25f7e566d4b5089c1b25b0b9ee9c5": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "31f7a65507b04611af0aeac4ab86a77f": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLStyleModel",
      "state": {
       "description_width": "",
       "font_size": null,
       "text_color": null
      }
     },
     "323a686079b5420f8d826df9375de3a2": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLModel",
      "state": {
       "layout": "IPY_MODEL_b261bb8564a741e78b9084def8493fa5",
       "style": "IPY_MODEL_e5e3c2a5c2504995939c140b9e11173d",
       "value": "Map: 100%"
      }
     },
     "35230867ec8c40559296037257ec87da": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "39b5c0733e4c48d9bb99a32a190fda66": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "FloatProgressModel",
      "state": {
       "bar_style": "success",
       "layout": "IPY_MODEL_35230867ec8c40559296037257ec87da",
       "max": 14041,
       "style": "IPY_MODEL_213c849d49074d2784eb4e9a749be116",
       "value": 14041
      }
     },
     "3d61676807034e878116577acf44c29e": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "FloatProgressModel",
      "state": {
       "bar_style": "success",
       "layout": "IPY_MODEL_a8991c542b994b6ba82c0ad154ff0fd9",
       "max": 3453,
       "style": "IPY_MODEL_b4855ab2cb534c5d898933e23be9dcd6",
       "value": 3453
      }
     },
     "3d6e0fc8b405457b9ad9e6bf154f26bd": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "description_width": ""
      }
     },
     "44496c25ff204a2289c63cb2cac51fa2": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLModel",
      "state": {
       "layout": "IPY_MODEL_162f22edce4446ae8ee2ad3c7e6afb98",
       "style": "IPY_MODEL_2f38221907b44d5db9af4896d07f21a4",
       "value": "Downloading data: 100%"
      }
     },
     "44ce8385e7934b36889614beba1ec74e": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HBoxModel",
      "state": {
       "children": [
        "IPY_MODEL_323a686079b5420f8d826df9375de3a2",
        "IPY_MODEL_2f4f1cee64e444c59655032f355f837b",
        "IPY_MODEL_ebe6cf40d613458184a07e8dbe20576e"
       ],
       "layout": "IPY_MODEL_08c6f36cda9f40e6a829764815a5d042"
      }
     },
     "476cc6c9cb3b4be796c9b004a6d6efcf": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "4d5580b2dc8d4dfabdbfa419c7ee465e": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "517101c6212f46ea881fff19fd840f07": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "FloatProgressModel",
      "state": {
       "bar_style": "success",
       "layout": "IPY_MODEL_1122e48971774d0db5bb72bdcb362a33",
       "max": 3250,
       "style": "IPY_MODEL_3d6e0fc8b405457b9ad9e6bf154f26bd",
       "value": 3250
      }
     },
     "5277e4bda0dd420ab87723f000fff2a8": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "description_width": ""
      }
     },
     "534443f19f4f4304964831cda942efa8": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "539cb75bf83d479bac698316801593a3": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HBoxModel",
      "state": {
       "children": [
        "IPY_MODEL_ea747956697b4ba7b65fbbf252bdde0e",
        "IPY_MODEL_7cd7dc1cdb2649d0aa2d2bd0506d328a",
        "IPY_MODEL_7defb149558c4e708af9244abc3d0a61"
       ],
       "layout": "IPY_MODEL_091e9d9766ce414f98bb52001a874d84"
      }
     },
     "5ad12bc9a9554621ace4ac9a663fb80c": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "5df269a6754f4c7ab69e07f28dd63569": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "6768402963504600b251ad3037bd91df": {
      "model_module": "@jupyter-widgets/output",
      "model_module_version": "1.0.0",
      "model_name": "OutputModel",
      "state": {
       "layout": "IPY_MODEL_5df269a6754f4c7ab69e07f28dd63569",
       "outputs": [
        {
         "data": {
          "text/html": "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Statistics collection <span style=\"color: #729c1f; text-decoration-color: #729c1f\">━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━</span> <span style=\"color: #800080; text-decoration-color: #800080\">100%</span> <span style=\"color: #0068b5; text-decoration-color: #0068b5\">100/100</span> • <span style=\"color: #0068b5; text-decoration-color: #0068b5\">0:00:02</span> • <span style=\"color: #0068b5; text-decoration-color: #0068b5\">0:00:00</span>\n</pre>\n",
          "text/plain": "Statistics collection \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[38;2;0;104;181m100/100\u001b[0m • \u001b[38;2;0;104;181m0:00:02\u001b[0m • \u001b[38;2;0;104;181m0:00:00\u001b[0m\n"
         },
         "metadata": {},
         "output_type": "display_data"
        }
       ]
      }
     },
     "6c727072b0cf4927a7d8930caaa35977": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLStyleModel",
      "state": {
       "description_width": "",
       "font_size": null,
       "text_color": null
      }
     },
     "73ebb155d1e748239d49f132e21c92cb": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "FloatProgressModel",
      "state": {
       "bar_style": "success",
       "layout": "IPY_MODEL_f62b1435601b456aba8263693359c2ba",
       "max": 9570,
       "style": "IPY_MODEL_2f8a65b6e630498893ac2f3655a70b89",
       "value": 9570
      }
     },
     "746da1704ae449158bf7e1606a5665f4": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "description_width": ""
      }
     },
     "7538b0a34110454fbfdf82b58a19ab58": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HBoxModel",
      "state": {
       "children": [
        "IPY_MODEL_212d64bbafdb4b37af03df36843297e5",
        "IPY_MODEL_3d61676807034e878116577acf44c29e",
        "IPY_MODEL_8eade49514f34288bbac997d26ba2c81"
       ],
       "layout": "IPY_MODEL_cc8f759ba1f74d7fa9f64ecafa083d17"
      }
     },
     "7664f24e88824df786ad118cf58933cd": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLModel",
      "state": {
       "layout": "IPY_MODEL_e656d2c7d2fd432a99e0a3ac1b2cca08",
       "style": "IPY_MODEL_f01afe29c481487c96b2bc48d39a0792",
       "value": " 14041/14041 [00:01&lt;00:00, 10455.43 examples/s]"
      }
     },
     "7863d5d193b44624aea46c8fc724ddf7": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLStyleModel",
      "state": {
       "description_width": "",
       "font_size": null,
       "text_color": null
      }
     },
     "7cd7dc1cdb2649d0aa2d2bd0506d328a": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "FloatProgressModel",
      "state": {
       "bar_style": "success",
       "layout": "IPY_MODEL_2e4b55f1c6d54fa8a6a40d1f8ccdfe52",
       "max": 1,
       "style": "IPY_MODEL_746da1704ae449158bf7e1606a5665f4",
       "value": 1
      }
     },
     "7defb149558c4e708af9244abc3d0a61": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLModel",
      "state": {
       "layout": "IPY_MODEL_0937816eac10417596e1337304c46a34",
       "style": "IPY_MODEL_6c727072b0cf4927a7d8930caaa35977",
       "value": " 1/1 [00:00&lt;00:00, 230.00it/s]"
      }
     },
     "82c66224007c475881297072b3fc303a": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "description_width": ""
      }
     },
     "82d5cfcc002a45d89e6f3045eb8fb298": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "description_width": ""
      }
     },
     "844190c85f2d4a438d4d7c8fe28266a3": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "87bf70d51a0d4c229e25ac179e3c3303": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLModel",
      "state": {
       "layout": "IPY_MODEL_476cc6c9cb3b4be796c9b004a6d6efcf",
       "style": "IPY_MODEL_b594875446334a089bfd88d80f19afc7",
       "value": "Generating validation split: 100%"
      }
     },
     "89af588477f54df7b5bb02f5fd24e8f9": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "8a52879e7ebb4daf8dc53f2499278ccb": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLModel",
      "state": {
       "layout": "IPY_MODEL_9a663a6139fe4af89d40508c453ae59b",
       "style": "IPY_MODEL_31f7a65507b04611af0aeac4ab86a77f",
       "value": "README.md: 100%"
      }
     },
     "8a93dd201c0440419f43ef30cce70751": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "8c91355c59354bddb22b0c5592d8529e": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLModel",
      "state": {
       "layout": "IPY_MODEL_a158b785fe0b4585ba228d2b236b689a",
       "style": "IPY_MODEL_fc428858726e470f964650b0babe10fe",
       "value": " 983k/983k [00:01&lt;00:00, 1.24MB/s]"
      }
     },
     "8d1c61591445470fa5c6a79fdb5c1e8c": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "8d575b4e3df843638deba33d294d0f67": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "8dc9b408fdd9474792f5288df9a30792": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "8e57931377a049829c15c0d708b14394": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "8eade49514f34288bbac997d26ba2c81": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLModel",
      "state": {
       "layout": "IPY_MODEL_251853a90016490abb9747f3b8d41c28",
       "style": "IPY_MODEL_a0e220e1417f4e7faaf8d96c711b4d8f",
       "value": " 3453/3453 [00:00&lt;00:00, 10079.43 examples/s]"
      }
     },
     "913c1b70c5794a6cbbaeee1a87df51c6": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLStyleModel",
      "state": {
       "description_width": "",
       "font_size": null,
       "text_color": null
      }
     },
     "9398e11d0c1744f18c35c3a13c7621d7": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "description_width": ""
      }
     },
     "99115728ba7d4e0c8f45aa1308c12430": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HBoxModel",
      "state": {
       "children": [
        "IPY_MODEL_44496c25ff204a2289c63cb2cac51fa2",
        "IPY_MODEL_3033be28482f4b8c90ead9f18345de66",
        "IPY_MODEL_8c91355c59354bddb22b0c5592d8529e"
       ],
       "layout": "IPY_MODEL_1ffe50db1bab4cd3a76ffce73d96755b"
      }
     },
     "9a663a6139fe4af89d40508c453ae59b": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "9a876a2e31614fb29efcc8abae021382": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLStyleModel",
      "state": {
       "description_width": "",
       "font_size": null,
       "text_color": null
      }
     },
     "a0e220e1417f4e7faaf8d96c711b4d8f": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLStyleModel",
      "state": {
       "description_width": "",
       "font_size": null,
       "text_color": null
      }
     },
     "a158b785fe0b4585ba228d2b236b689a": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "a538094363694cf499345dbd4838c7a7": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLModel",
      "state": {
       "layout": "IPY_MODEL_c1b64cb041f940fcb3d46d4d23e385c7",
       "style": "IPY_MODEL_c39fdb20b51546cabc9bdb77e956662f",
       "value": "Generating train split: 100%"
      }
     },
     "a6fcb65daf7a4ba58330deb79c321832": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLModel",
      "state": {
       "layout": "IPY_MODEL_5ad12bc9a9554621ace4ac9a663fb80c",
       "style": "IPY_MODEL_913c1b70c5794a6cbbaeee1a87df51c6",
       "value": " 12.3k/12.3k [00:00&lt;00:00, 1.27MB/s]"
      }
     },
     "a8991c542b994b6ba82c0ad154ff0fd9": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "a97fe8ae0b184a7e9316a11ed67078b9": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "aa676eefd46d464d9d605262f52a1864": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HBoxModel",
      "state": {
       "children": [
        "IPY_MODEL_a538094363694cf499345dbd4838c7a7",
        "IPY_MODEL_39b5c0733e4c48d9bb99a32a190fda66",
        "IPY_MODEL_7664f24e88824df786ad118cf58933cd"
       ],
       "layout": "IPY_MODEL_163cb33d39c444a09f7d99c2ffa93f4d"
      }
     },
     "aefdd06f6777415c8eb76e53a479fd66": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLModel",
      "state": {
       "layout": "IPY_MODEL_8d1c61591445470fa5c6a79fdb5c1e8c",
       "style": "IPY_MODEL_df5c3bb1846443c88af27ce96fe8a0ba",
       "value": " 3250/3250 [00:00&lt;00:00, 5119.37 examples/s]"
      }
     },
     "b14b2069a3c94631adae48670c0b2869": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HBoxModel",
      "state": {
       "children": [
        "IPY_MODEL_8a52879e7ebb4daf8dc53f2499278ccb",
        "IPY_MODEL_c1288b77e0694507926f1c98f032dd0a",
        "IPY_MODEL_a6fcb65daf7a4ba58330deb79c321832"
       ],
       "layout": "IPY_MODEL_241ee2f446604688a29d27da13e94eba"
      }
     },
     "b261bb8564a741e78b9084def8493fa5": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "b4855ab2cb534c5d898933e23be9dcd6": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "description_width": ""
      }
     },
     "b594875446334a089bfd88d80f19afc7": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLStyleModel",
      "state": {
       "description_width": "",
       "font_size": null,
       "text_color": null
      }
     },
     "b5bc109b202142b3969d8a1849d3a74f": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "ba3e07f964b140e09a1234d74dac1524": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLStyleModel",
      "state": {
       "description_width": "",
       "font_size": null,
       "text_color": null
      }
     },
     "bbcb27fb663a42a1980c29ef7b0c7db2": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "c1288b77e0694507926f1c98f032dd0a": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "FloatProgressModel",
      "state": {
       "bar_style": "success",
       "layout": "IPY_MODEL_8d575b4e3df843638deba33d294d0f67",
       "max": 12330,
       "style": "IPY_MODEL_e70216b099b14571b450e70f03f29c3d",
       "value": 12330
      }
     },
     "c1b64cb041f940fcb3d46d4d23e385c7": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "c39fdb20b51546cabc9bdb77e956662f": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLStyleModel",
      "state": {
       "description_width": "",
       "font_size": null,
       "text_color": null
      }
     },
     "c6a418bdc6bb4b558c26b44564ee589b": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HBoxModel",
      "state": {
       "children": [
        "IPY_MODEL_2c964cf3312e4ab5800cce91c5f51ed4",
        "IPY_MODEL_73ebb155d1e748239d49f132e21c92cb",
        "IPY_MODEL_e839bccdec0c4f70be3d08d09002ee48"
       ],
       "layout": "IPY_MODEL_844190c85f2d4a438d4d7c8fe28266a3"
      }
     },
     "cc8f759ba1f74d7fa9f64ecafa083d17": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "d058b3c141ab48bbaf483a99d918f967": {
      "model_module": "@jupyter-widgets/output",
      "model_module_version": "1.0.0",
      "model_name": "OutputModel",
      "state": {
       "layout": "IPY_MODEL_4d5580b2dc8d4dfabdbfa419c7ee465e",
       "outputs": [
        {
         "data": {
          "text/html": "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Applying Smooth Quant <span style=\"color: #729c1f; text-decoration-color: #729c1f\">━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━</span> <span style=\"color: #800080; text-decoration-color: #800080\">100%</span> <span style=\"color: #0068b5; text-decoration-color: #0068b5\">25/25</span> • <span style=\"color: #0068b5; text-decoration-color: #0068b5\">0:00:00</span> • <span style=\"color: #0068b5; text-decoration-color: #0068b5\">0:00:00</span>\n</pre>\n",
          "text/plain": "Applying Smooth Quant \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[38;2;0;104;181m25/25\u001b[0m • \u001b[38;2;0;104;181m0:00:00\u001b[0m • \u001b[38;2;0;104;181m0:00:00\u001b[0m\n"
         },
         "metadata": {},
         "output_type": "display_data"
        }
       ]
      }
     },
     "df5c3bb1846443c88af27ce96fe8a0ba": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLStyleModel",
      "state": {
       "description_width": "",
       "font_size": null,
       "text_color": null
      }
     },
     "e49cc0736d4f493cbb95e9c4fadf1e73": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "e5e3c2a5c2504995939c140b9e11173d": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLStyleModel",
      "state": {
       "description_width": "",
       "font_size": null,
       "text_color": null
      }
     },
     "e656d2c7d2fd432a99e0a3ac1b2cca08": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "e70216b099b14571b450e70f03f29c3d": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "description_width": ""
      }
     },
     "e839bccdec0c4f70be3d08d09002ee48": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLModel",
      "state": {
       "layout": "IPY_MODEL_b5bc109b202142b3969d8a1849d3a74f",
       "style": "IPY_MODEL_ee2f0a98e2e146b5855fa6da363f48df",
       "value": " 9.57k/9.57k [00:00&lt;00:00, 1.05MB/s]"
      }
     },
     "ea747956697b4ba7b65fbbf252bdde0e": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLModel",
      "state": {
       "layout": "IPY_MODEL_a97fe8ae0b184a7e9316a11ed67078b9",
       "style": "IPY_MODEL_7863d5d193b44624aea46c8fc724ddf7",
       "value": "Computing checksums: 100%"
      }
     },
     "ebe6cf40d613458184a07e8dbe20576e": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLModel",
      "state": {
       "layout": "IPY_MODEL_e49cc0736d4f493cbb95e9c4fadf1e73",
       "style": "IPY_MODEL_fd52d32432ff4fd98a3292dd25dd056f",
       "value": " 100/100 [00:00&lt;00:00, 4183.09 examples/s]"
      }
     },
     "ee2f0a98e2e146b5855fa6da363f48df": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLStyleModel",
      "state": {
       "description_width": "",
       "font_size": null,
       "text_color": null
      }
     },
     "f01afe29c481487c96b2bc48d39a0792": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLStyleModel",
      "state": {
       "description_width": "",
       "font_size": null,
       "text_color": null
      }
     },
     "f1020d70684d4ca6bc9658d85834185d": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "DropdownModel",
      "state": {
       "_options_labels": [
        "CPU",
        "AUTO"
       ],
       "description": "Device:",
       "index": 1,
       "layout": "IPY_MODEL_8e57931377a049829c15c0d708b14394",
       "style": "IPY_MODEL_82d5cfcc002a45d89e6f3045eb8fb298"
      }
     },
     "f62b1435601b456aba8263693359c2ba": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "f802d1cf73ce46efa90dba7e52ddced7": {
      "model_module": "@jupyter-widgets/output",
      "model_module_version": "1.0.0",
      "model_name": "OutputModel",
      "state": {
       "layout": "IPY_MODEL_8dc9b408fdd9474792f5288df9a30792",
       "outputs": [
        {
         "data": {
          "text/html": "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">Applying Fast Bias correction <span style=\"color: #729c1f; text-decoration-color: #729c1f\">━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━</span> <span style=\"color: #800080; text-decoration-color: #800080\">100%</span> <span style=\"color: #0068b5; text-decoration-color: #0068b5\">37/37</span> • <span style=\"color: #0068b5; text-decoration-color: #0068b5\">0:00:01</span> • <span style=\"color: #0068b5; text-decoration-color: #0068b5\">0:00:00</span>\n</pre>\n",
          "text/plain": "Applying Fast Bias correction \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[35m100%\u001b[0m \u001b[38;2;0;104;181m37/37\u001b[0m • \u001b[38;2;0;104;181m0:00:01\u001b[0m • \u001b[38;2;0;104;181m0:00:00\u001b[0m\n"
         },
         "metadata": {},
         "output_type": "display_data"
        }
       ]
      }
     },
     "f8f9441945794f949e84109564504f60": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "DropdownModel",
      "state": {
       "_options_labels": [
        "CPU",
        "AUTO"
       ],
       "description": "Device:",
       "index": 1,
       "layout": "IPY_MODEL_fab0d6712f334970a626f67a030fa764",
       "style": "IPY_MODEL_82c66224007c475881297072b3fc303a"
      }
     },
     "fab0d6712f334970a626f67a030fa764": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {}
     },
     "fc428858726e470f964650b0babe10fe": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLStyleModel",
      "state": {
       "description_width": "",
       "font_size": null,
       "text_color": null
      }
     },
     "fd52d32432ff4fd98a3292dd25dd056f": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLStyleModel",
      "state": {
       "description_width": "",
       "font_size": null,
       "text_color": null
      }
     }
    },
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
